Classification by Polynomial Surfaces

Linear threshold functions (for real and Boolean inputs) have received much attention, for they are the component parts of many artificial neural networks. Linear threshold functions are exactly those functions such that the positive and negative examples are separated by a hyperplane. One extension of this notion is to allow separators to be surfaces whose equations are polynomials of at most a given degree (linear separation being the degree-1 case). We investigate the representational and expressive power of polynomial separators. Restricting to the Boolean domain, by using an upper bound on the number of functions defined on {0, 1}n by polynomial separators having at most a given degree, we show, as conjectured by Wang and Williams [26], that for almost every Boolean function, one needs a polynomial surface of degree at least bn/2c in order to separate the negative examples from the positive examples. Further, we show that, for odd n, at most half of all Boolean functions are realisable by a separating surface of degree bn/2c. We then compute the Vapnik-Chervonenkis dimension of the class of functions realised by polynomial separating surfaces of at most a given degree, both for the case of Boolean inputs and real inputs. In the case of linear separators, the VC dimensions coincide for these two cases, but for surfaces of higher degree, there is a strict divergence. We then use these results on the VC dimension to quantify the sample size required for valid generalisation in Valiant’s probably approximately correct framework [24, 6].